Skip to content

Consolidate PAFTOOLS into one task per sample - #26

Merged
jpalmer37 merged 1 commit into
mainfrom
refactor/optimize-paftools-workflow
Oct 5, 2026
Merged

jpalmer37 merged 1 commit into
mainfrom
refactor/optimize-paftools-workflow

Conversation

@jpalmer37

Copy link
Copy Markdown
Collaborator

Same pass as #25, for PAFTOOLS. After BLAST, each sample now runs one paftools_variants task (modules/paftools/paftools.nf) in place of nine. Per-sample tasks go from 11 to 3: blastn, paftools_variants, mutation_watch. minimap2 gets an accession-only reference, so the CHROM rename before SnpEff, the reference sed in post_process_vcf and bcftools sort are gone.

Review modules/paftools/paftools.nf and workflows/paftools.nf first. The rest is cleanup:

  • filter_alignments.py: drop the dead groupby, the unused --min_cov/--min_id/--metric, and the min-max composite. Reference selection is unchanged; the composite crashed under pandas 3 when every hit had the same score.
  • Remove blastn's unused emits and split_fasta.py combine.
  • post_process_vcf drops from 16 to 2 CPUs.

Output changes:

  • Coverage TSV reference_id is the bare accession, not acc|subtype|segment.
  • A segment with no BLAST reference gets a FAIL coverage row.
  • Provenance has one paftools_variants record.
  • *_blastn_filter.tsv loses composite.

Verified on 6 GISAID samples (H1N1/H5N1/H9N2 SnpEff databases), running FASTA prep + PAFTOOLS through Nextflow on main and on this branch. All 48 final VCFs match apart from ##SnpEffCmd/##bcftools_sort header lines, and every other published file matches. Tasks went from 92 to 44 and summed PAFTOOLS task time from 370 s to 153 s. pytest has the same 5 test_run_snpeff.py failures as main. SHORT_READ was not smoke-tested.

Fold split, align, coverage QC, call, sort, SnpEff, conversion and MNV
correction into paftools_variants, and drop dead code along the path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jpalmer37
jpalmer37 marked this pull request as ready for review October 5, 2026 20:55
@jpalmer37
jpalmer37 merged commit 373630a into main Oct 5, 2026
1 check failed
@jpalmer37
jpalmer37 deleted the refactor/optimize-paftools-workflow branch October 15, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant